Encoding device and method, decoding device and method, editing device and method, recording medium, and program

ABSTRACT

The present invention relates to an encoding device and a method, a decoding device and a method, an editing device and a method, a storage medium, and a program which can perform encoding and decoding so that buffer failure does not occur. Information, such as a minimum bit rate, a minimum buffer size, and a minimum initial delay time, is contained in a random access point header contained in an accessible point in a bitstream. A bitsteam analyzing unit  72  analyzes an input bitstream, sets the above-mentioned information, and outputs the resulting information to a buffer-information adding unit  73.  The buffer-information adding unit  73  adds the input information to the input bitstream and outputs the resulting bitstream. The present invention is applicable to an encoding device and a decoding device which process bitstreams.

CROSS REFERENCE TO RELATED APPLICATION

The present continuation application claims the benefit of priorityunder 35 U.S.C. §120 to U.S. application Ser. No. 14/293,553, filed Jun.2, 2014, which is a continuation of U.S. application Ser. No.12/401,476, filed on Mar. 10, 2009, which is a division of U.S.application Ser. No. 10/510,701 filed Oct. 15, 2004, which is a NationalStage of PCT/JP03/04598, filed Apr. 11, 2003, and claims the benefit ofpriority under 35 U.S.C. §119 of Japanese Application No. 2002-125298,filed Apr. 26, 2002. The entire contents of which are incorporatedherein by reference.

TECHNICAL FIELD

The present invention relates to encoding devices and methods, decodingdevices and methods, editing devices and methods, storage media, andprograms. In particular, the present invention relates to an encodingdevice and a method, a decoding device and a method, an editing deviceand a method, a storage medium, and a program which are preferably usedfor transmitting/receiving image information (bitstreams), compressed bymotion compensation and orthogonal transformation, such as a discretecosine transform or Karhunen-Loeve transform, through a network medium,such as satellite broadcast, cable television broadcast, or theInternet, or are preferably used for processing image information on astorage medium, such as an optical disk, a magnetic disk, or a flashmemory.

BACKGROUND ART

In recent years, devices that digitally process image information withan aim of efficient transmission and storage of information and that arecompliant with an MPEG (Moving Picture Expert Group) standard or thelike for compression by motion compensation and orthogonaltransformation, such as a discrete cosine transform, using animage-information-specific redundancy are becoming widespread in bothinformation distribution by broadcast stations and information receptionby general homes.

In particular, MPEG-2 (ISO/IEC 13818-2) is a standard that was definedas a universal image compression scheme and that covers both interlacedscan images and progressive scan images as well as standard resolutionimages and high-definition images. MPEG-2 is widely used in a widevariety of applications for professional and consumer applications, forexample, as represented by a DVD (Digital Versatile Disk) standard.

Through the use of the MPEG-2 compression scheme, for example, assigninga bit rate (bit rate) of 4 to 8 Mbps to a standard-resolution interlacedscan image having 720×480 pixels and a bit rate of 18 to 22 Mbps to ahigh-resolution interlaced scan image having 1920×1088 pixels canachieve favorable images with high compression ratios.

MPEG-2 was mainly intended for high-quality coding suitable forbroadcasting, but was not compatible with a coding scheme employing ahigher compression ratio and thus an MPEG-4 coding scheme has beenstandardized. With regard to an image coding scheme, the scheme wasapproved as an ISO/IEC 14496-2 international standard in December, 1998.

In addition, in recent years, with an initial purpose of image codingfor videoconferences, the standardization of what is called H.26L (ITU/TQ6/16 VCEG) by ITU-T (International TelecommunicationUnion-Telecommunication Standardization Sector) is in progress. H.26Lrequires a larger amount of computation for encoding and decoding,compared to MPEG-2 and MPEG-4 coding schemes, but is known as achievinghigher coding efficiency.

Further, as part of MPEG-4 activities, the standardization of a codingtechnique for realizing higher coding efficiency based on H.26L iscurrently underway by JVT (Joint Video Team) in cooperation with ITU-T.

Now, a description is given of image compression using motioncompensation and orthogonal transformation, such as a discrete cosinetransform or Karhunen-Loeve transform. FIG. 1 is a diagram showing theconfiguration of one example of a conventional image-informationencoding device.

In an image-information encoding device 10 shown in FIG. 1, imageinformation provided by an analog signal input from an input terminal 11is converted by an A/D converter 12 into a digital signal. The screenrearranging buffer 13 rearranges frames in accordance with a GOP (Groupof Pictures) structure of image information supplied from the A/Dconverter 12.

Here, with respect to an image on which intra (intra-image) encoding isperformed, the screen rearranging buffer 13 supplies image informationof an entire frame to an orthogonal transformation unit 15. Theorthogonal transformation unit 15 performs a discrete cosine transformor Karhunen-Loeve transform on the image information and supplies atransform coefficient to a quantization unit 16. The quantization unit16 performs quantization processing on the transform coefficientsupplied from the orthogonal transformation unit 15.

Based on a quantization scale and a transform coefficient quantized andsupplied by the quantization unit 16, a reversible encoding unit 17determines an encoding mode, performs variable-length encoding orreversible encoding, such as arithmetic encoding, on the encoding modeto create information to be inserted into the header portion of an imageencoding unit. The reversible encoding unit 17 then supplies the encodedencoding-mode to a storage buffer 18 for storage. The encodedencoding-mode is output from an output terminal 19 as compressed imageinformation.

The reversible encoding unit 17 also performs variable-length encodingor reversible encoding, such as arithmetic encoding, on the quantizedtransform coefficient and supplies the encoded transform coefficient tothe storage buffer 18 for storage. The encoded transform coefficient isoutput from the output terminal 19 as compressed image information.

The behavior of the quantization unit 16 is controlled by a ratecontroller 20 in accordance with the amount of data of transformcoefficient stored in the storage buffer 18. The rate controller 20 alsosupplies a quantized transform coefficient to a dequantization unit 21.The dequantization unit 21 dequantizes the quantized transformcoefficient. An inverse orthogonal transformation unit 22 performsinverse orthogonal transformation processing on the dequantizedtransform coefficient to generate decoded image information and suppliesthe information to a frame memory 23 for storage.

With respect to an image on which inter (inter-image) encoding isperformed, the screen rearranging buffer 13 supplies image informationto a motion prediction/compensation unit 24. The motionprediction/compensation unit 24 simultaneously retrieves imageinformation that is referred to from the frame memory 23 and performsmotion prediction/compensation processing on the image information togenerate reference image information. The motion prediction/compensationunit 24 supplies the generated reference image information to an adder14. The adder 14 converts the reference image information into a signalindicating a difference relative to corresponding image information. Atthe same time, the motion prediction/compensation unit 24 also suppliesmotion vector information to the reversible encoding unit 17.

The reversible encoding unit 17 determines an encoding mode from thequantization scale and the transform coefficient quantized and suppliedby the quantization unit 16 and the motion vector information suppliedfrom the motion prediction/compensation unit 24. The reversible encodingunit 17 performs variable-length encoding or reversible encoding, suchas arithmetic encoding on the determined encoding mode to generateinformation to be inserted into the header portion of an image encodingunit. The reversible encoding unit 17 supplies the encoded encoding-modeto the storage buffer 18 for storage. The encoded encoding-mode isoutput as compressed image information.

The reversible encoding unit 17 performs variable-length encoding orreversible encoding processing, such as arithmetic encoding, on themotion vector information to generate information to be inserted intothe header portion of an image encoding unit.

Unlike intra encoding, in the case of inter encoding, image informationinput to the orthogonal transformation unit 15 is a difference signalprovided by the adder 14. Since other processing is analogous to thatfor the compressed image information on which intra encoding isperformed, the description thereof is omitted.

Next, the configuration of one embodiment of an image-informationdecoding device corresponding to the above-described image-informationencoding device 10 will be described with reference to FIG. 2. In animage-information decoding device 40 shown in FIG. 2, compressed imageinformation input from an input terminal 41 is temporarily stored by astorage buffer 42 and is then transferred to a reversible decoding unit43.

In accordance with a predetermined compressed-image-information format,the reversible decoding unit 43 performs processing, such as variablelength decoding or arithmetic decoding, on the compressed imageinformation. The reversible decoding unit 43 then obtains encoding modeinformation stored in a header portion and supplies the encoding modeinformation to a dequnatization unit 44. Similarly, the reversibledecoding unit 43 obtains a quantized transform coefficient and suppliesthe coefficient to the dequnatization unit 44. When a frame to bedecoded has been subjected to inter encoding, the reversible decodingunit 43 also decodes motion vector information stored in the headerportion of compressed image information and supplies the information toa motion prediction/compensation unit 51.

The dequnatization unit 44 dequantizes the quantized transformcoefficient supplied from the reversible decoding unit 43 and suppliesthe resulting transform coefficient to an inverse orthogonaltransformation unit 45. In accordance with a predeterminedcompressed-image-information format, the inverse orthogonaltransformation unit 45 performs inverse orthogonal transformation, suchas inverse discrete cosine transform or Karhunen-Loeve transform, on thetransform coefficient.

Here, when a frame of interest has been subjected to intra encoding,image information subjected to the inverse orthogonal transformationprocessing is stored in a screen rearranging buffer 47. After the imageinformation is subjected to D/A conversion processing by a D/A converter48, the resulting information is output from an output terminal 49.

Also, when a frame of interest has been subjected to inter encoding, themotion prediction/compensation unit 51 generates a reference image inaccordance with motion vector information that has been subjected toreversible decoding processing and image information stored in a framememory 50, and supplies the reference image to an adder 46. The adder 46combines the reference image with the output from the inverse orthogonaltransformation unit 45. Since other processing is analogous to that forthe frame that has been subjected to intra encoding, the descriptionthereof is omitted.

For a coding scheme (hereinafter referred to as “JVT Codec”) beingstandardized by the above-mentioned Joint Video Team, various schemeshave been under consideration in order to improve the coding efficiencyof MPEG-2, MPEG-4, and so on. For example, for the transformation schemeof a discrete cosine transform, an integer-coefficient transform with a4×4 block size is used. Further, the block size for motion compensationis variable, so that more optimum motion compensation can be performed.The basic scheme, however, can be realized in the same manner as theencoding scheme performed in the image-information encoding device 10shown in FIG. 1.

Thus, the JVT codec can perform decoding using essentially the samescheme as the decoding scheme performed in the image-informationdecoding device 40 shown in FIG. 2.

Meanwhile, in order to maintain the compatibility between differentencoding devices (decoders) and to prevent buffer overflow or underflow,The MPEG and ITU-T use a buffer model. A virtual decoder buffer model isstandardized and an encoding device (encoder) performs encoding so thatthe virtual decoder buffer does not fail. This makes it possible toprevent buffer overflow or underflow at the decoder side and to maintainthe compatibility.

A virtual buffer model according to the MPEG will be described withreference to FIG. 3. In the following description, R indicates an inputbit rate for a decoder buffer, B indicates the size of the decoderbuffer, F indicates the amount of buffer occupied when the decoderextracts a first frame from the buffer, and D indicates delay timetherefor.

Bit amounts of each frame at time t0, t1, t2, are indicated by b0, b2,b2 . . . , and so on.

When the frame rate is M, the following expression is satisfied:

t _(i+1) −t _(i)=1/M

When B_(i) indicates the amount of buffer occupancy immediately beforebit amount b_(i) of a frame at time t_(i) is extracted, expression (1)below is satisfied:

B₀=F

B _(i+1)=min(B, B _(i) b _(i) +R(t _(i+1) −t _(i)))   (1)

In this case, for a fixed bit rate encoding scheme for MPEG-2, theencoder must perform encoding so as to satisfy condition (2) below:

Bi≦B

Bi−bi≧0   (2)

As long as such a condition is satisfied, the encoder should not performencoding that causes buffer overflow and underflow.

Further, for a variable bit rate encoding scheme for MPEG-2, the inputbit rate R is a maximum bit rate defined by a profile and a level and isgiven by F=B. Thus, expression (1) can be rewritten as expression (3)

B₀=B

B _(i+1)=min(B, B _(i) −b _(i) +R(t _(i+1) −t _(i)))   (3)

In this case, the encoder must perform encoding so as to satisfyexpression (4) below:

B _(i) −b _(i)≧0   (4)

When this condition is satisfied, the encoder will perform encoding thatdoes not cause buffer underflow at the decoder side. When the decoderbuffer becomes full, the encoder buffer is empty and this indicates thatno encoding bitstream is generated. Thus, there is no need for theencoder to perform monitoring so that the buffer overflow of the decoderdoes not occur.

In the MPEG, encoding is performed so as to comply with theabove-described buffer restrictions in accordance with a buffer size anda bit rate defined by each profile and level. Thus, a decoder thatconforms to each profile and level can perform decoding without causingfailure of the bitstream.

In practice, however, without the use of a buffer size and a bit ratedefined by a profile and a level, there are cases in that a bitstreamcan be decoded.

For example, a bitstream encoded with a bit rate R, a buffer B, andinitial delay time F, i.e., (R, B, F), can be decoded by a decoderhaving a larger buffer size B′ (B′>B). The bitstream can also be decodedat a higher bit rate R′ (R′>R).

For example, when the decoding bit rate of a decoder is lower than anencoding bit rate, a decoder that has a sufficiently large buffer sizecan perform decoding.

In this manner, when a predetermined bitstream is given, a minimumbuffer size B_(min) needed to decode the bitstream exists at each bitrate. Such a relationship is shown in FIG. 4.

The standardization of JVT Codec has been in progress so that decodingis possible not only with a fixed bit rate and a buffer size defined byeach profile and level but also is possibly by a decoder having thecondition shown in FIG. 4. This has the objective of allowing decodingeven if the decoding bit rate and buffer size of an encoder and thedecoding bit rate and buffer size of a decoder are not necessarily thesame. By achieving the objective, for example, a decoder having a highdecoding bit rate can reduce a buffer size.

However, such information varies in a bitstream with time. Thus, thereis a problem in that, even when decoding is possible under apredetermined condition, decoding may be impossible under anothercondition, since the restrictions for decoder compatibility has beenrelaxed. For example, when such a characteristic of (R, B) varies withtime, there is a problem in that, even when decoding is possible atpredetermined time, decoding may be impossible at another time.

Further, there is a problem in that decoding is not always possible inthe case of shifting to another scene or another channel due to randomaccess or the like. There is also a problem in that the decodingpossibility cannot be guaranteed when bitstream-level editing, such assplicing (splicing), is performed.

DISCLOSURE OF INVENTION

The present invention has been made in view of such situations, and anobject of the present invention is to efficiently determine the decodingpossibility of a bitstream and to simplify bitstream editing, such assplicing.

An encoding device of the present invention includes generating meansfor generating header to which reference is made as needed duringdecoding; encoding means for encoding the header generated by thegenerating means and an input image signal, respectively; and outputtingmeans for multiplexing the header and the image signal encoded by theencoding means and outputting a bitstream. The encoding device ischaracterized in that the generating means generates the headercontaining buffer characteristic information about buffering duringdecoding of the bitstream.

The generating means can generate the header containing the buffercharacteristic information for each predetermined section randomlyaccessible in the bitstream.

The generating means can generate the header containing the buffercharacteristic information for an entire sequence of the bitstream.

The buffer characteristic information can contain all of a minimum bitrate, a minimum buffer size B, and a minimum delay amount which aredecodable during decoding of the bitstream.

An encoding method of the present invention includes a generating stepof generating a header to which reference is made as needed duringdecoding; an encoding step of encoding the header generated in thegenerating step and an input image signal, respectively; and an outputcontrolling step of controlling output of a bitstream in which theheader and the image signal encoded by the processing in the encodingstep are multiplexed. The encoding method is characterized in that theprocessing in the generating step generates the header containing buffercharacteristic information about buffering during decoding of thebitstream.

A program for a first storage medium of the present invention includes agenerating step of generating a header to which reference is made asneeded during decoding; an encoding step of encoding the headergenerated by the processing in the generating step and an input imagesignal, respectively; and an output controlling step of controllingoutput of a bitstream in which the header and the image signal encodedby the processing in the encoding step are multiplexed. The program ischaracterized in that the processing in the generating step generatesthe header containing buffer characteristic information about bufferingduring decoding of the bitstream.

A first program of the present invention causes a computer to executeprocessing that includes a generating step of generating a header towhich reference is made as needed during decoding; an encoding step ofencoding the header generated in the generating step and an input imagesignal, respectively; and an output controlling step of controllingoutput of a bitstream in which the header and the image signal encodedby the processing in the encoding step are multiplexed. The program ischaracterized in that the processing in the generating step generatesthe header containing buffer characteristic information about bufferingduring decoding of the bitstream.

A decoding device of the present invention is characterized by includingsearching means for searching for a header in an input bitstream; anddecoding means for reading buffer characteristic information aboutbuffering, which information contained in the header found by thesearching means, and for decoding the bitstream in accordance with theread buffer characteristic information.

The buffer characteristic information can contain all of a minimum bitrate, a minimum buffer size, and a minimum delay amount which aredecodable during decoding of the bitstream.

A decoding method of the present invention is characterized by includinga searching step of searching for a header in an input bitstream; and adecoding step of reading buffer characteristic information aboutbuffering, which information contained in the header found by theprocessing in the searching step, and of decoding the bitstream inaccordance with the read buffer characteristic information.

A program for a second storage medium of the present invention ischaracterized by including a searching step of searching for a header inan input bitstream; and a decoding step of reading buffer characteristicinformation about buffering, which information contained in the headerfound by the processing in the searching step, and of decoding thebitstream in accordance with the read buffer characteristic information.

A second program of the present invention is characterized by causing acomputer to execute processing that includes a searching step ofsearching for a header in an input bitstream; and a decoding step ofreading buffer characteristic information about buffering, whichinformation contained in the header found by the processing in thesearching step, and of decoding the bitstream in accordance with theread buffer characteristic information.

An editing device of the present invention includes searching means forsearching for a header in an input bitstream; determining means forreading buffer characteristic information about buffering, whichinformation contained in the header found by the searching means, andfor determining whether or not the bitstream can be edited in accordancewith the read buffer characteristic information; and editing means forediting the bitstream when the determining means determines that thebitstream can be edited. The editing device is characterized in that,when a characteristic curve created from the information contained inthe header in a first bitstream is always located above or is the sameas a characteristic curve created from the information contained in theheader in a second bitstream, the determining means determines thatediting using the first bitstream and the second bitstream is possible.

An editing method of the present invention includes a searching step ofsearching for a header in an input bitstream; a determining step ofreading buffer characteristic information about buffering, whichinformation contained in the header found by the processing in thesearching step, and of determining whether or not the bitstream can beedited in accordance with the read information; and an editing step ofediting the bitstream when the processing in the determining stepdetermines that the bitstream can be edited. The editing method ischaracterized in that, when a characteristic curve created from theinformation contained in the header in a first bitstream is alwayslocated above or is the same as a characteristic curve created from theinformation contained in the header in a second bitstream, theprocessing in the determining step determines that editing using thefirst bitstream and the second bitstream is possible.

A program for a third storage medium of the present invention includes asearching step of searching for a header in an input bitstream; adetermining step of reading buffer characteristic information aboutbuffering, which information contained in the header found by theprocessing in the searching step, and of determining whether or not thebitstream can be edited in accordance with the read information; and anediting step of editing the bitstream when the processing in thedetermining step determines that the bitstream can be edited. Theprogram is characterized in that, when a characteristic curve createdfrom the information contained in the header in a first bitstream isalways located above or is the same as a characteristic curve createdfrom the information contained in the header in a second bitstream, theprocessing in the determining step determines that editing using thefirst bitstream and the second bitstream is possible.

A third program of the present invention causes a computer to executeprocessing that includes a searching step of searching for a header inan input bitstream; a determining step of reading buffer characteristicinformation about buffering, which information contained in the headerfound by the processing in the searching step, and of determiningwhether or not the bitstream can be edited in accordance with the readinformation; and an editing step of editing the bitstream when theprocessing in the determining step determines that the bitstream can beedited. The program is characterized in that, when a characteristiccurve created from the information contained in the header in a firstbitstream is always located above or is the same as a characteristiccurve created from the information contained in the header in a secondbitstream, the processing in the determining step determines thatediting using the first bitstream and the second bitstream is possible.

According to the encoding device and the method as well as the firstprogram of the present invention, buffer characteristic informationabout buffering during decoding of a bitstream is contained in a headerthat is encoded and multiplexed with the bitstream. This can prevent adecoding side from causing buffer failure.

According to the decoding device and the method as well as the secondprogram of the present invention, buffer characteristic informationabout buffering during decoding, which information contained in a headerin an input bitstream, is read and decoding is performed in accordancewith the read information.

According to the editing device and the method as well as the thirdprogram of the present invention, a determination as to whether an inputbitstream can be edited is made by determining whether a characteristiccurve created from information contained in a header in a firstbitstream is always located above or is the same as a characteristiccurve created from information contained in a header in a secondbitstream.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram showing the configuration of one example of aconventional image-information encoding device.

FIG. 2 is a diagram showing the configuration of one example of aconventional image-information decoding device.

FIG. 3 is a graph for describing the amount of buffer.

FIG. 4 is a graph for describing the relationship between a bit rate andthe amount of buffer.

FIG. 5 is a diagram showing the configuration of one embodiment of anencoding device according to the present invention.

FIG. 6 is graph for describing the amount of buffer.

FIG. 7 is a diagram showing the configuration of one embodiment of adecoding device according to the present invention.

FIG. 8 is a diagram showing the configuration of one embodiment of anediting device according to the present invention.

FIG. 9 is a graph for describing the relationship between a bit rate andthe amount of buffer.

FIG. 10 is a diagram for describing a medium.

BEST MODE FOR CARRYING OUT THE INVENTION

An embodiment of the present invention will be described below withreference to the accompanying drawings. FIG. 5 is a diagram showing theconfiguration of one embodiment of an encoding device according to thepresent invention. An encoding device 70 shown in FIG. 5 includes theimage-information encoding device 10 shown in FIG. 1. Here, theconfiguration and so on of the image-information encoding device 10 hasbeen described, the description thereof is omitted as appropriate.

Image information input to the image-information encoding device 10 isencoded and is output as compressed image information (BS: bitstream) toa buffer 71 and a bitstream analyzing unit 72. The buffer 71 temporarilystores the input bitstream and outputs the bitstream to abuffer-information adding unit 73 as needed. The bitstream analyzingunit 72 checks the buffer occupancy state of a predetermined section ina bitstream, for example, a section between GOPs or random accesspoints, and supplies the information to the buffer-information addingunit 73 as buffer information BH. Here, the “random access point” refersto a predetermined section randomly accessible in a bitstream in a JVTstandard. Similarly, the “GOP” refers to a predetermined sectionrandomly accessible in an MPEG-2/MPEG-4 standard.

The buffer-information adding unit 73 adds the input buffer informationBH to an input bitstream and outputs the resulting information.

In this case, as one example of analysis performed by the bitstreamanalyzing unit 72, a description is given of an exemplary case in whicha buffer occupancy state is checked between random access points and theinformation of the buffer occupancy state is encoded as headerinformation for each random access point to thereby constitute abitstream. While the description here is described in such a manner, theencoding may be performed in units of GOP or another arbitrarily unitmay be used. Thus, needless to say, the present invention is applicableto a case in which another unit is used in stead of a unit describedbelow.

A method for determining the characteristic of (R_(min), B_(min)) willbe described with reference to FIG. 6. R_(min) herein indicates theminimum value of an input bit rate R for the buffer and B_(min)indicates the minimum value of a buffer size B.

When the bit rate R of a predetermined bitstream is given, the minimumbuffer size B_(min) that is decodable by a decoding device (e.g., havinga configuration shown in FIG. 7) for decoding the bitstream at thedecoding bit rate R is determined, for example, in the following manner.

The number of frames between predetermined access points is indicated byN. The amount of bits generated for each frame is b(i) (i=1, N), theamount of buffer occupancy immediately before data of each frame isextracted from the buffer is B(i), and the amount of buffer occupancyimmediately after the extraction is B2(i). The amount of buffer of theencoding device is indicated by B. Then, the following is given:

B2(i)=B(i)−b(i)

B(i+1)−B2(i)+R/(Frame Rate)   (5)

where if (B(i+1)>B)B(i+1)=B and the maximum value of B(i) is B. Further,suppose a delay amount F satisfies F=B.

In this case, B_(min) can be determined by expression (6) below:

B _(min) =B−min(B2(i))   (6)

When R is assumed to be R_(min) in this case, the above described methodcan determine (R_(min), B_(min)).

Next, one example of a method for determining (R_(min), B_(min),F_(min)) will be described. Let B=B_(min), R=R_(min). As in expression(5), expression (7) below is satisfied

B2(i)=B(i)−b(i)

B(i+1)=B2(i)+R/(Frame Rate)   (7)

where monitoring for an underflow is performed in accordance with thefollowing condition.

if (B2(i)<0){F _(min) =F _(min)+(0−B2(i)); B2(i)=0;}

F_(min) is initialized to “0” at the start of each random access point.Monitoring for an overflow is similarly performed in accordance with thefollowing condition.

if (B(i+1)>B)B(i+1)=B

(R_(min), B_(min), F_(min)) is determined by performing theabove-described checking on every frame between random access points.

(R_(min), B_(min), F_(min)) described above may be checked by apredetermined number thereof or may be defined by only independentcombinations thereamong. The characteristic determined as describedabove is as shown in FIG. 4. Portions between points are linearlyinterpolated. The values of (R_(min), B_(min), F_(min)) obtained asdescribed above, i.e., buffer information BH, are inserted into apredetermined position in a bitstream by the buffer-information addingunit 73, are encoded, and are output.

Simultaneously with (R_(min), B_(min), F_(min)) between random accesses,as described above, the bitstream analyzing unit 72 performs similaranalysis on the entire bitstream to determine the characteristic of theentire bitstream, i.e., (R_(min), B_(min), F_(min)) global. Thebitstream analyzing unit 72 then supplies the values thereof to thebuffer-information adding unit 73 as the buffer information BH.

The bitstream BS output from the image-information encoding device 10 isdelayed by a predetermined amount of time by the buffer 71 and is theninput to the buffer-information adding unit 73. The buffer-informationadding unit 73 inserts the buffer information BH, supplied from thebitstream analyzing unit 72, into a predetermined position in thebitstream and outputs a final output bitstream BS.

In this case, the buffer information BH (or buffer characteristicinformation) is, for example, (R_(min), B_(min), F_(min)) and (R_(min),B_(min), F_(min)) global. The buffer-information adding unit 73 insertsthe above-described information into a predetermined position in thebitstream BS. One example of syntax will be described below.

RAP_header ( ) {   RAP_startcode;   closed_GOP;   Broken_link;  NumBufferParam;   for(i=0;i<NumBufferParam;i++){      Rate[i];     Buffer[i];      F[i];}}

(R_(min), B_(min), F_(min)) between random access points is recorded in,for example, in a random access point header immediately therebefore asthe above-noted syntax. “RAP-startcode” is a code that indicates thepresence of a RAP header and that indicates the start of the header.

“closed_GOP” is a flag indicating whether either all pictures in the GOPare independent without referring to any picture of another GOP or aredependent on a picture in another GOP while referring thereto.“broken_link” is a flag indicating whether or not a reference image forprediction exits when a bitstream is replaced in front of or behind theGOP by editing or the like.

NumBuffer_Param indicates the number of determined characteristic sets(R_(min), B_(min), F_(min)). Rate[i], Buffer[i], and F[i] indicateR_(min), B_(min), and F_(min), respectively. In this case, for example,R_(min) is recorded in the increasing order.

(R_(min), B_(min) F_(min)) global of an entire bitstream is recorded in,for example, the first sequence header of the bitstream, as in thefollowing syntax:

Sequence_header ( ) {   Sequence_startcode;   :   :   NumBufferParam;    for(i=0;i<NumBufferParam;i++){       Rate[i];       Buffer[i];      F[i];}}

where NumBuffer_Param indicates the number of determined characteristicsets (R_(min), B_(min), F_(min)) global. Rate[i], Buffer[i], and F[i]indicate R_(min), B_(min), and F_(min), respectively. In this case,R_(min) is recorded, for example in the increasing order.

After attaching the above-described buffer information BH, thebuffer-information adding unit 73 outputs a final output bitstream BS.

In the embodiment of the present invention, as the buffer informationBH, the minimum bit rate Rmin, the minimum buffer size Bmin, and theminimum delay amount Fmin are all added to a bitstream in the abovedescription. However, the present invention is not limited to thatexample, and thus at least one of the minimum bit rate Rmin, the minimumbuffer size Bmin, and the minimum delay amount Fmin may be added to abitstream. For example, a combination of the minimum bit rate Rmin andthe minimum buffer size Bmin may be added to a bitstream.

FIG. 7 shows one embodiment of a decoding device according to thepresent invention. A decoding device 90 shown in FIG. 7 corresponds tothe encoding device 70 shown in FIG. 5. The decoding device 90 includesthe image-information decoding device 40 shown in FIG. 2 therein. Abitstream BS input to the decoding device 90 is supplied to a bitstreamanalyzing unit 91 a decoding-possibility determining unit 92.

The bitstream analyzing unit 91 decodes buffer information BH in thebitstream and outputs the resulting buffer information BH to thedecoding-possibility determining unit 92. The bitstream analyzing unit91 parses the bitstream to decode (R_(min), B_(min), F_(min))globalrecorded in the sequence header. The bitstream analyzing unit 91 alsodecodes (R_(min), B_(min), F_(min)) recorded in each random access pointheader. Those pieces of information are output to thedecoding-possibility determining unit 92.

The decoding-possibility determining unit 92 determines whether or notthe input bitstream can be decoded without the input bitstream causingbuffer failure, in accordance with the buffer information BH and decoderinformation DI supplied from the image-information decoding device 40.Examples of the decoder information DI include a decoder buffer size anda decoding bit rate.

The decoding-possibility determining unit 92 creates a characteristiccurve as shown in FIG. 4, based on (R_(min), B_(min), F_(min)) global.Portions between points are linearly interpolated. In this case, abuffer of the decoder (decoding device 90) and the decoding bit rate arelocated above the characteristic curve formed from (R_(min), B_(min),F_(min)) global, it is possible to determine that the input bitstream isdecodable. Thus, in such a case, the decoding-possibility determiningunit 92 determines that the input bitstream is decodable and suppliesthe bitstream to the image-information decoding device 40.

The image-information decoding device 40 has essentially the sameconfiguration as the image-information decoding device 40 shown in FIG.2, and executes similar processing to decode the input bitstream andoutputs image information to, for example, a television receiver whichis not shown.

Whether or not an entire bitstream is decodable can be determined bychecking the characteristic curve of (R_(min), B_(min), F_(min)) global,a decoder buffer size, and a decoding bit rate, as described above.

Also, when decoding of only a specific section from a predeterminedrandom access point is desired due to random accessing, thedecoding-possibility determining unit 92 similarly creates acharacteristic curve as shown in FIG. 4 from (R_(min), B_(min),F_(min)). Portions between points are linearly interpolated. In thiscase, when the decoder buffer and the decoding bit rate are locatedabove the characteristic curve created from (R_(min), F_(min)), thebitstream is decodable. Thus, in such a case, the decoding-possibilitydetermining unit 92 determines that the bitstream is decodable andsupplies the bitstream to the image-information decoding device 40.

Next, bitstream editing will be described. FIG. 8 is a diagram showingthe configuration of one embodiment of an editing device 110 for editinga bitstream according to the present invention. As an example of editingperformed by the editing device 110, a description is given of a case inwhich splicing is performed to replace a portion of an input bitstream 1with another input bitstream 2.

Now, splicing will be briefly described. Splicing is to perform editingby replacing a predetermined bitstream with another bitstream at arandom access point. Such splicing is performed, for example, when acommercial broadcast is inserted into a television broadcast program. Inthis case, the input bitstream 1 corresponds to a bitstream of thetelevision broadcast program and the input bitstream 2 corresponds to abitstream of the commercial.

The input bitstream 1 is input to a bitstream analyzing unit 111-1 andthe input bitstream 2 is input to a bitstream analyzing unit 111-2. Thebitstream analyzing units 111-1 and 111-2 decode buffer information BH1and BH2, contained in the input bitstreams 1 and 2, respectively, andoutput the resulting information to a bitstream editing unit 112.

In accordance with the buffer information BH1 and BH2, the bitstreamediting unit 112 determines whether or not the input bitstream 2 can beinserted into the input bitstream 1 at a predetermined editing point. Inthis case, in order that an edited bitstream is decodable without abuffer failure of the decoder (the decoding device 90), it is necessaryto satisfy a condition that the value of amount of buffer occupancy forthe random access point and the value of amount of buffer occupancy fora portion immediately before the point are the same.

Decoders using MPEG-2 or -4 were designed so as to operate at a specificbit rate and a buffer size. On the other hand, with regard to decodersusing a J VT scheme, restrictions for buffering has been relaxed so thatdecoding is possible when the characteristic curve is located above thecharacteristic curve of (R_(min), B_(min), F_(min)) even for other bitrates and buffer sizes, as shown in FIG. 4.

In order that bitstream editing does not cause the decoding possibilityto be varied before and after the editing, it is sufficient to have thesame (R_(min), B_(min), F_(min)) for the edit section. Thus, withrespect to the input bitstreams 1 and 2, the bitstream editing unit 112creates the characteristic (R_(min), B_(min), F_(min)) for a randomaccess point header located in the edit section, and replaces thesection with the bitstream 2 when those values match each other. Whenthe values do not match each other, the bitstream editing unit 112inserts padding bits to the bitstream 1 or 2 so that the values of(R_(min), B_(min), F_(min)) match each other and then replaces acorresponding section with the input bitstream 2.

Restrictions for buffering have been relaxed in the JVT, and the use ofthis advantage makes it possible to ease the buffer compatibilitycondition in splicing. In the JVT, when the decoder buffer size and thedecoding bit rate are located above (R_(min), B_(min), F_(min)), it canbe known that the bitstream is decodable. Thus, when (R_(min), B_(min),F_(min)) of a predetermined edit section of the input bitstream 2 to beinserted is always located below (R_(min), B_(min), F_(min)) of apredetermined edit section of the original input bitstream 1, a decoderthat can decode the input bitstream 1 can also perform decoding evenwith that section being replaced with the bitstream 2.

FIG. 9 illustrates the relationship. Curve 1 represents thecharacteristic of (R_(min), B_(min), F_(min)) of an edit section of theinput bitstream 1. Curve 2 represents the characteristic of (R_(min),B_(min), F_(min)) of an edit section of the input bitstream 2. When thedecoder buffer and the decoding bit rate are located above the curve,the bitstream is decodable. Thus, when curve 2 is always located belowcurve 1 as shown in FIG. 9, it is guaranteed that decoding is possible.

Accordingly, the bitstream editing unit 112 creates the characteristicof (R_(min), B_(min), F_(min)) for a random access point header locatedin an edit section with respect to the bitstreams 1 and 2. Then, whenthe characteristic curve of the bitstream 2 is located below thecharacteristic curve of the bitstream 1, the bitstream editing unit 112replaces the corresponding section with the bitstream 2.

Conversely, when the characteristics do not match, the bitstream editingunit 112 makes a change by inserting padding bits to the bitstream 1 or2 so that the characteristic curve of (R_(min), B_(min), F_(min)) of thebitstream 2 is located below the characteristic curve of the bitstream1, and then replaces the section with the input bitstream 2.

When splicing is performed so as to satisfy such condition, a decoderthat can decode the bitstream 1 will not fail. After splicing, thebitstream editing unit 112 outputs a final bitstream.

In this manner, containing such information (R_(min), B_(min), F_(min)),namely, a minimum bit rate, minimum buffer size, and minimum initialdelay time, in the header of a randomly accessible point in a bitstreamallows the decoding side to efficiently determine the decodingpossibility of the bitstream. Further, this arrangement can facilitatebitstream editing, such as splicing, and can always decode the bitstreamwithout causing buffer failure of a decoding side.

FIG. 10 is a diagram showing an example of the internal configuration ofa general-purpose personal computer. A CPU (Central Processing Unit) 211of the personal computer executes various processing in accordance witha program stored in a ROM (Read Only Memory) 212. As appropriate, a RAM(Random Access Memory) 213 stores data and/or a program required for theCPU 211 to execute various processing. An input/output interface 215 isconnected to an input unit 216, which is constituted by a keyboard andmouse, and outputs a signal, input through the input unit 216, to theCPU 211. The input/output interface 215 is also connected to an outputunit 7, which is constituted by a display, speakers, and so on.

Further, the input/output interface 215 is connected to a storage unit218, which includes a hard disk and so on, and a communication unit 219,which communicates data with another device through a network such asthe Internet.

A drive 220 is used to write/read data to/from a storage medium, such asa magnetic disk 231, an optical disk 232, a magneto-optical disk 233, ora semiconductor memory 234.

As shown in FIG. 10, the storage medium is not only implemented by apackage medium distributed separately from a personal computer to supplya program to a user, but is also implemented by a hard disk includingthe ROM 212 or the storage unit 218 that stores a program and that issupplied to a user after being pre-installed on a computer. An exampleof the package medium in which a program is recorded is the magneticdisk 231 (including a flexible disk), the optical disk 232 (includingCD-ROM (Compact Disc-Read Only Memory), a DVD (Digital Versatile Disc),and the magneto-optical disk 233 (including an MD (Mini-Disc)(registered trademark)), or a semiconductor memory 234.

Herein, steps for writing the program supplied with a medium not onlyinclude processing that is time-sequentially performed according to theorder described above but also include processing that is notnecessarily performed time-sequentially but is executed in parallel orindependently.

Herein, the system represents an entire device that is constituted by aplurality of devices.

INDUSTRIAL APPLICABILITY

As described above, according to the encoding device and the method aswell as the first program of the present invention, buffercharacteristic information about buffering during decoding of abitstream is contained in a header that is encoded and multiplexed withthe bitstream. This can prevent a decoding side from causing bufferfailure.

According to the decoding device and the method as well as the secondprogram of the present invention, buffer characteristic informationabout buffering during decoding, which information contained in a headerin an input bitstream, is read and decoding is performed in accordancewith the read information. This can prevent buffer failure during thedecoding.

Further, according to the editing device and the method as well as thethird program of the present invention, a determination as to whether aninput bitstream can be edited is made by determining whether acharacteristic curve created from information contained in a header in afirst bitstream is always located above or is the same as acharacteristic curve created from information contained in a header in asecond bitstream. This makes it possible to reduce processing requiredfor editing, such as splicing, and to determine whether editing isreadily possible.

1. (canceled)
 2. An encoding method, comprising: setting identificationinformation identifying whether either all pictures in a predeterminedsection randomly accessible and buffer characteristics information,which includes a plurality of combinations of buffer size informationand bit rate information as criteria for determining whether a bitstreamis decodable using a decoder buffer size and a decoding bit rate, thebuffer size information indicating a required buffer size of a bufferthat stores the bitstream during decoding of the bitstream and the bitrate information indicating an input bit rate of the buffer; andencoding an image signal to generate the bitstream including theidentification information and the buffer characteristics information.3. An encoding device comprising: circuitry configured to setidentification information identifying whether either all pictures in apredetermined section randomly accessible and buffer characteristicsinformation, which includes a plurality of combinations of buffer sizeinformation and bit rate information as criteria for determining whethera bitstream is decodable using a decoder buffer size and a decoding bitrate, the buffer size information indicating a required buffer size of abuffer that stores the bitstream during decoding of the bitstream andthe bit rate information indicating an input bit rate of the buffer; andencode an image signal to generate the bitstream including theidentification information and the buffer characteristics information.